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Abstract: Free and hybrid models of multisource satellite images 
fusion are developed using the plausible and paradoxical reasoning 
theory of Dezert-Smarandache. The aim of this work is to show 
the contribution of these fusion models for improving the thematic 
classification and the quantification of change. The maps obtained by 
the free model are composed by simple classes and compound classes. 
Nevertheless, they contain no significant thematic classes and require 
an important computing time. In the other hand, the hybrid model 
with a constraint introduced using a prior knowledge relatively of the 
study area, can have maps composed of more realistic classes in a 
reduced time. These models are implemented and tested on images 
acquired by SPOT HRV and Landsat ETM+ sensors. 
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22.1 Introduction 


Recently, the number of satellite sensors is growing. Information acquired by the 
various satellite sensors is very rich and complementary. The combination or the 
fusion of different types of information become very interesting. It must take into 
account the sources of information increasingly numerous and varied. Information 
fusion resulting from different sources remains an open and important problem. The 
difficulty of this process is due to both uncertain and conflicting information available. 

In this context, several approaches and theories have been developed [2, 5, 13, 17]. 
The probabilistic approach which is the oldest and most widespread, it can represent 
well the uncertainty in the information but does not represent its imprecision [2]. 
Moreover, it reasons on only simple classes that represent different hypothesis. How- 
ever, the Dempster-Shafer Theory (DST) can be an alternative to the probabilistic 
approach, it is often recommended and used by some authors [2, 3, 10-12] because it 
can also put up with the uncertain nature of information through a solid mathematical 
formalism and Dempster combination rule. 

Nevertheless, this theory has certain weaknesses when the combined evidence 
sources become very conflicting (conflict close to the unit) and when the problem to 
be processed cannot be directly described within the frame of discernment of this 
theory due the paradoxical nature of information. Consequently a new theory which 
can be considered as a generalization of DST was elaborate, it is the plausible and 
paradoxical reasoning theory of Dezert-Smarandache (DSmT) [6, 7, 13, 14], it was 
applied in the field of remote sensing by [3, 4]. This theory can solve some delicate 
problems where DST is usually fails. 

DSmT starts with the notion of free DSm model. This model is free because no 
other assumption is done on the hypotheses. When the free DSm model holds, the 
classic commutative and associative DSm rule of combination is performed. In this 
free model, the rule of combination takes into account both uncertain and paradoxical 
information. Thus, it generates a frame of discernment more general. But, if the 
cardinal of this frame increases the computing time increases and moreover some 
classes of the power set are not significant. Therefore, a integrity constraints are 
explicitly and formally introduced into the free DSm model in order to adapt it 
properly to fit as close as possible with the reality and permit to construct the hybrid 
model. There exist actually many possible hybrid models between the two extreme 
models (Shafer model and free model) for the frames depending on the real intrinsic 
nature of elements. The hybrid DSm rule works in any model and is involved in 
calculating the combined mass of any number of information sources, no matter 
how big is the conflict /paradoxism of sources, and on any frame (exhaustive or non- 
exhaustive, with elements which may be exclusive or non-exclusive or both) [13]. 

The aim of our work is the improvement of the thematic classification and the 
quantification of changes by a fusion process of optical satellite images using two 
models of DSmT (the free and the hybrid models). These images are covering a zone 
of study located at the east of Algiers. 
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The remainder of the paper is organized as follows. In the next section, we recall 
the mathematical basis of DSmT and its application to fusion process. Section 22.3 is 
devoted to the presentation and the implementation of the free model and the hybrid 
model of DSmT. In section 22.4, the two models of DSmT will be applied to the 
fusion of two multisource and multitemporal images. Finally, section 22.5 gathers 
our conclusions and the possible prospects to this work. 


22.2 Dezert Smarandache theory basis 


The DSmT of plausible, uncertain and paradoxical reasoning [6, 8, 9, 13, 15] is a 
generalization of the classical DST [5, 16] which allows to formally combine any 
types of sources of information (rational, uncertain or paradoxical). The DSmT 
is able to solve complex data/information fusion problems where the DST usually 
fails, especially when conflicts (paradoxes) between sources become large and when 
the refinement of the frame of discernment © is inaccessible because of the vague, 
relative and imprecise nature of O elements. The foundation of DSmT is based on 
the definition of the hyper-power set D® (Dedekind’s lattice) of a general frame of 
discernment © [8, 9]. The foundation of DSmT is based on the definition of the 
hyper-power set D®° [8, 9] which detailed in section 1.2.1 of the chapter 1, in the 
beginning of this book. 


22.2.1 Mass functions 


The determination of mass functions in DSmT represents a crucial step in a fusion 
process and remains a largely unsolved problem, which did not yet find a general 
answer. In image processing, Bloch [2] describes three different levels from where a 
mass function may be derived: at the highest level where information representation 
is used in a way similar to that in artificial intelligence and masses are assigned to 
propositions; at an intermediate level, masses are computed from attributes, and may 
involve simple geometrical models; at the pixel level, mass assignment is inspired 
from statistical pattern recognition. Recall that the difficulty increases when we 
are interested on the compound hypotheses and their mass functions. The most 
widely used approach is to assign to simple hypotheses masses that are computed 
from conditional probabilities. Then a transfer model is introduced to distribute 
the initial masses over all compound hypotheses (union and intersection of classes). 
This transfer operation is done through a coarsening (discounting) factor and/or a 
conditioning factor applying to the conditional probabilities (initial masses). 


In this paper, the mass functions are estimated using a dissonant model of Appriou 
that was initially developed for only two classes [1] and we have generalized and 
extrapolated for more than two classes as follows [3]. In the following equations, x 
stands for the value of a pixel of the SPOT or Landsat image at spectral band 6 and 
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spatial location s. 
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where & is the number of the considered classes, ¢ is a sensitivity factor that weighted 
the mass functions in order to have their sum over all the hypothesis equal to 1, 
P(«?|6;) is the conditional probability, a? is a coarsening factor, and R? represents a 
normalization factor that is introduced in the axiomatic approach in order to respect 
the mass and plausibility definitions, and is given by: 
1 

MaxXj=1,...,k P(a|6;) , 
To fuse paradoxical or rational sources of information (bodies of evidence), we have 
used in this paper the DSm classical rule and the DSm hybrid rule. These rules are 
detailed in the sections 1.2.4 and 1.2.5 of the chapter 1 in this book. 

For a future work, we plan to test the PCR5 for the multi-source satellite image 
fusion. This rule redistributes every partial conflict only to propositions only which 
are truly involved in it and proportionally to their masses put in the conflict [15]. 


22.2.2 Decision Rule 


After the combination of different sources, a decision is made according to a certain 
criteria. Several decision rules have been proposed: 


1. maximum of plausibility which is advocated by some authors [2-4, 11, 12], 
2. maximum of belief over the simple hypothesis which is the most used [11], 


3. maximum of belief without overlapping of belief intervals which is very strict 
and called absolute decision rule [3, 11, 12], 


4. maximum of pignistic probability [13, 17]. 


22.3. Implementation of the free and hybrid models 


22.3.1 Implementation of the free model 


The fusion process with the free model M/(@) is given in Fig. 22.1. 
The fusion process is detailed by the following steps: 
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Figure 22.1: Multi-source fusion process using the free model. 


1. A geometrical correction in the same reference frame using the interpolation 
by the polynomial model. 


2. A radiometric corrections for both images. 


3. According to an a prior knowledge, two data bases are constructed: a training 
base to be used in a supervised classification process, and a test base to be used 
during the assessment of the classification accuracy. 


4. A Bayesian classification is performed using a maximum likelihood algorithm. 


5. A confusion matrix is established between a Bayesian classification result and 
a test data base. 


6. For each class, a coarsening factor is obtained from the confusion matrix and 
it can be seen as the accuracy of that class which is computed by dividing the 
total number of correct pixels in that class by each of the total number of pixels 
in that category as derived from the test data base. 


7. The mass function is estimated using transfer model of Appriou, detailed in 
sec. 22.2.1. 
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The mass function is estimated one more time by using transfer model of Ap- 
priou. 


A combination rule of DSmT between sources is applied to obtain the combined 
mass 5; given by: 


S,(A) 4 > [[ mx) (22.2) 


Ra, MajucXpeD? t=1 
X{NX9N...0X,=A 


The combined mass 1 is saved. 


The belief and the plausibility functions are deduced from the combined mass 
function. 


The uncertainty for each pixel is calculated. 


Finally, a multispectral classification is released according to a decision rule. 


22.3.2 Implementation of the hybrid model 


The fusion process with the hybrid model M(@) is given in Fig. 22.2 and detailed by 
the following steps. 


1. 


2. 


Introduction of the combined mass S$ calculated in the free model M/(@). 
Introduction of the constraints by forcing some elements of D©® to be empty. 


Determination of the characteristic non-emptiness function ¢(A) and the total 
empty set @ = {Om, O}. 


Calculation of the sum based on the technique of absorption, transferred the 
mass from each empty element to total or relative ignorance using the expres- 
sion of S2 (see section 1.2.5 in chapter 1) given by: 


S2(A) > [[ mx) (22.3) 
X1,Xo0,....X,E0 i=l 
(U=A]v[(UE9)A(A=Iz)] 


Calculation of the sum transferred the masses of relative empty sets to nonempty 
sets using the following expression of S3 (see section 1.2.5 in chapter 1): 


k 
S3(A) & eS [[ mi) (22.4) 
X1,XQ,...,X,ED? t=1 
X1{UXQU...UX,=A 
X1{NX_gN...9X,EO 
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6. Calculation of the combined masses using the general rule of hybrid combina- 
tion of DSm [7] defines as follows: 


VA; € D°, muey(Ai) = (As) [Marro) (Ai) + $2(Ai) + S3(Aa)] - 


7. The belief and the plausibility functions are deduced from the combined mass 
function. 


8. The uncertainty for each pixel is calculated. 


9. Finally, a multispectral classification is released according to a decision rule. 
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Figure 22.2: Multi-source fusion process using the hybrid model. 
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22.4 Application 


22.4.1 Site of study and data used 


The methodology proposed is tested on an area located approximately at 10km to 
the east of Algiers. This area is characterized by high urban concentration and a very 
dense road network in the north of the airport and an agricultural area with bare soil 
in the south of the airport. 

For a multisource study of the site, we often used a data set acquired by different 
satellite sensors at the same date on the same study area and for a multitemporal 
study of the site, it is preferable to use a data set acquired by the same sensor on 
different times on the same scene. However, currently we do not have this ideal data 
set. Therefore, two multisource and multitemporal images were put at contribution 
in this study: a multispectral image acquired on April 1°, 1997 by the sensor HRV1 
of SPOT-1 satellite, a multispectral image acquired on June 3'¢, 2001 by the sensor 
ETM- of Landsat-7 satellite. The joint exploitation of these images requires a step 
of geo-referencing through a method of geometric correction. 


In our case, we applied the polynomial method through a second order polynomial. 
Then, we proceeded to the resampling of HRV image at a resolution of 30 m using 
the method of Nearest Neighbour. Thus, the RGB compositions of the two images 
corrected are shown in Fig. 22.3. 


(a) SPOT HRV 1997 (b) Landsat ETM+ 2001 


Figure 22.3: RGB composition of the Algiers scene, Algeria. 
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The methodology of fusion and classification adopted in this work is supervised 
based on a prior knowledge on the study site and the various themes which are there. 
Then, we extracted a training base and a test base for each image. These bases 
contain three thematic classes: Dense Urban (DU), Bare Soil (BS) and Vegetation 
(V) that have been identified and defined by an expert knowing well this area of 
study. 

The validity of the choice of the three classes for the steps of training and evalu- 
ation is carried out and justified in [10]. Indeed, for the two images, we notice that 
the difference between the envelope of the three normal distributions associated with 
the three classes and the form of the real histogram is very negligible. This means 
that the two images are dominated by the three classes considered. 


22.4.2 Fusion based on the free model 


In a multitemporal study of a site, it is preferable to use a multitemporal data set 
acquired by the same satellite sensor on the same study area. However, we do not 
have this data set. Therefore, two multisource and multitemporal images were used 
in this study. 


The improvement of the land cover maps obtained is based on the joint exploita- 
tion of the two essential characteristics of the sensors which provide the images. The 
first characteristic is the wealth of the spectral information of the image acquired by 
sensor ETM+ (six spectral bands) which allows a better identification and discrim- 
ination of the themes on the ground, and the second characteristic is the wealth of 
the spatial information of the image acquired by the sensor HRV (spatial resolution 
of 20 m) which allows a more detailed description of the objects. 


The result of multisource classification and fusion obtained by the free model is 
given by Fig. 22.4. 

The evaluation of this result will focus only on the invariant sites between the 
two dates of acquisition (1997 and 2001). The airport’s runways are considered as in- 
variant site and have not undergone any changes between the two dates of acquisition. 


We note, from Fig. 22.4, that the multisource image obtained by the free model 
and more exactly the sites invariant of the airport’s runways, constitute of the simple 
classes DU on which the two sensors of acquisition (ETM+ and HRV) give the same 
opinion with certainty, and of the compound classes (intersection of classes) like the 
classes UM V, (BS UV)MU and BSN V, on which the two sensors give different 
opinions, i.e., there is a confusion between the two sensors. 


By taking a pixel located in the airport’s runways, belonging to the class of in- 
tersection UNM V generated by the free model, we note that its spectral signature in 
image HRV corresponds to the signature of the class U, and that its signature in 
image ETM-+ thus corresponds to the signature of the class V, the attribution of this 
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Figure 22.4: Result of multi-source fusion based on the free model. U: Urban 
area, V: Vegetation, BS: Bare soil. 


pixel to the class UNM V is well justified (see Fig.22.5). 


(a) Class Urban (U) (b) Class Vegetation (V) 


Figure 22.5: Spectral signatures of the classes Urban (HRV 1997) and Veg- 
etation (ETM+ 2001) in the invariant site of airport’s runways by the free 
model. 


The evaluation of the result in the case of multitemporal fusion will carry only 
on the sites varying between the two dates (1997 and 2001). We take as an example 
of variant sites, an agricultural zone located at the south of the airport. 
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We note that the multitemporal image obtained by the free model in more exactly 
the variant sites, constitute of simple classes representing the stable zones as the class 
V not having undergone any change, and of the compound classes representing the 
zones of changes during the time considered, as the class of intersection BSN V which 
is an unstable zone. 


By taking a pixel, located in the agricultural zone, belonging to the class BSN V 
generated by our methodology, we notice that its spectral signature in image HRV 
corresponds to the signature of the class BS, and that its signature in image ETM+ 
corresponds to the signature of the class V (Fig. 22.6). Therefore, the attribution of 
this pixel which changed class BS towards V to the class BSN V is well justified. 


(a) Class Bare Soil (BS) (b) Class Vegetation (V) 


Figure 22.6: Spectral signatures of the classes Bare soil (HRV 1997) and Veg- 
etation (ETM+ 2001) in the variant site of the agricultural zone by the free 
model. 


The result of the binary changes detection between 1997 and 2001 by multisource 
and multitemporal classification and fusion using the free model is given in Fig. 22.7. 
The simple classes represent the no change (in black), on the other hand the com- 
pound classes represent the change (in white). 


From a qualitative evaluation of this image, we see that there are a great dynamics 
in the study area between the two dates considered, an evolution of the thematic 
classes “bare soil” and “vegetation” in the south of the airport which is due one side, 
to the clearing of the land agricultural and another side to the farm of the bare areas 
and a dense urbanization in the north of the airport, in particular in the area of El 
Hamiz. 
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Figure 22.7: Binary changes image between 1997 and 2001 obtained by the free 
model. 


22.4.3 Fusion based on the hybrid model 


From a prior knowledge on the study area, we take as constraint, the proposition: 
UNV. Therefore, the set of the focal elements of D® is reduced to the following set: 


{U, BS, V, UUBS, UU V, BSU V, UUBSUV, UN BS, BSNV, 
(UU V) NBS, (UNBS) UV, (BSNV) UU}. 


Decision making will be done on the simple classes and the classes of intersection, by 
neglecting the masses associated to the unions of classes which are very weak. These 
classes are: 

{U,BS,V, UN BS, BSN V,(UUV) NBS}. 


The result of multisource classification and fusion based on the hybrid model is given 
by Fig. 22.8. 


The evaluation of the result obtained by multisource fusion using the hybrid model 
will always focus to the invariant sites between the dates 1997 and 2001. We note that 
the multisource image obtained by the hybrid model (©) and more exactly in the 
airport’s runways constitutes of pure classes as the class U on which the two sensors 
give a common opinion and the class of intersection as the class BS 7 V on which 
the two sensors give a different opinion. This result is well illustrated by the trace 
of the spectral signatures (Fig. 22.9) of a pixel of airport’s runways the belonging to 
the class of intersection BSN V. 

In multitemporal fusion, we see that there is a change of themes which has oc- 
curred on the agricultural zone in the south of the airport. A great change of the 
Bare Soil (origin) towards Vegetation (destination). The validation of this result is 
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Figure 22.8: Result of multi-source fusion based on the hybrid model. 


(a) Class Bare Soil (BS) (b) Class Vegetation (V) 


Figure 22.9: Spectral signatures of the classes bare Soil (HRV 1997) and Vege- 
tation (ETM+ 2001) in the invariant site of the airport’s runway by the hybrid 
model. 


done by taking a pixel belonging to the class BSM V and then, to observe its variation 
between 1997 and 2001. 


From the spectral signatures of Fig. 22.10, we see that a pixel of the class “Bare 
Soil” in this variant site in 1997 changed class after four years towards the simple 
class “Vegetation”. 

In the multisource and multitemporal fusion using the hybrid model, the binary 
changes map obtained is the same one as for the free model, that is due to the decision 
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(a) Class Bare Soil (BS) (b) Class Vegetation (V) 


Figure 22.10: Spectral signatures of the classes Bare Soil (HRV 1997) and 
Vegetation (ETM+ 2001) in the variant site of the agricultural zone by the 
hybrid model. 


rule which we applied. The pixels which represent no change (simple classes) are the 
pixels which belong to the same simple class in the two results obtained by Maximum 
Likelihood (ML). On the other hand, the pixels which represent the change are the 
pixels which belong to compound classes. 


The only difference between both change maps is at the level of the compound 
classes. In case of the free model, the number of change classes is greater than the 
number of change classes in case of the hybrid model. 


22.4.4 Comparison between the free and hybrid models 


After having obtained the land cover map and changes map using the free and hybrid 
models of DSmT, we carried out a comparative study between these two models. The 
various results of this study are listed on Table 22.1. 


22.5 Conclusion 


Multisource classification using the free model of DSmT presents an image composed 
from simple classes on which both acquisition sensors (ETM + and HRV) express 
the same opinion, and compound classes (intersection of classes) on which the two 
sensors express different opinions, relatively to the multitemporal classification that 
provides a changes map composed of simple classes representing stable areas which 
have not undergone any change, and compound classes represent the change areas 
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Size of memory needed mportant and for n 2 


Computing time Acceptable 


6 insufficient Sufficient 


; ; Includes non-significant | Includes more realistic 
Obtained image 
classes classes 


Table 22.1: Comparison between the free model and the hybrid model. 


during the time considered. To obtain these results, we require much computing 
time. On the other hand, the hybrid model allows to have maps composed of classes 
more significant and concordant with the ground reality. The results obtained will 
be exploited in cartography. 

We propose as possible prospects for our work: the integration within the fu- 
sion/classification process different types of satellite data known as heterogeneous for 
example: the contextual information or a satellite image from SAR (Synthetic Aper- 
ture Radar) to include topographic information or relief of the surface to classify for a 
more realistic and optimal, the use of the recent data and the update of the training 
base, the use of other rules of combination such as the PCR5 (Proportional Con- 
flict Redistribution), URR (Uniform Redistribution Rule), PURR (Partially Uniform 
Redistribution Rule). 
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